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SECTION I. 
Introduction 


A. Load Balance in Cloud Computing 


Cloud computing is an Internet-based resource utility [1], [2]. Its central concept is “everything can 
be a service”. In cloud computing, computing hardware and software resources are capsulized as 
web-services that can be accessed through the Internet. Three application types of cloud computing 
models are defined [3]: infrastructure as a service (IaaS), platform as a service (PaaS), and software 
as a service (SaaS). Among them, virtualization is a representative IaaS application, which provides 
computing infrastructure resources, such as computing power, data storage, networking, all in the 
form of web services [4], [5]. IaaS providers purchase and maintain physical computing and 
storage hardware and provide web services to users. With the virtualization technology, the users 
request IaaS providers for computation or storage resources as they own a “virtual machine”(VM) 
without purchasing and maintaining physical hardware. The users can utilize VMs for deploying 
system/application software with a considerably lower cost of hardware procurement and 
possession. 


An IaaS provider operates a server farm consisting of some computing and storage hardware, 
wherein some hosting servers (referring to as VMHs) provide virtualization services. A VMH may 
run one or many VMs, depending on the capacity of the VMH. If VMHs are not properly managed in 
a server farm, some VMHs may be busy running many VMs but some VMHs almost idle with few 
VMs. Managing loads of VMHs by adjusting the consumption of resources held by VMs for better 
cost-performance efficiency while guaranteeing service level agreements (SLA) to the customers is 


an essential issue in IaaS [6]- [9]. 


Migration of VMs among VMHs is one of the methods for load balance of VMHs, which is to move 
the workload of VMs from one overloaded VMH to other VMHs, trying to make the workload of all 
VMHs evenly. Three phases of work are involved in this process: detection, decision, and action. 


PTEE a is to detect if an imbalance occurs in a server farm. The decision phase is to 
Loading [MathJax]/jax/output/HTML-CSS/aytol lej ; r : e P 
pading [MathJax]jaxsutpu aga L eis gration of VMs is needed and select the VMs for migration and VMHs for accepting 
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„these VMs. The action phase is to suspend the VMs selected, migrate them from one VMH to others, 
Bana restart them after the migration. During the migration, only the Ms QARtABSA are 
De¥tispended and restarted, and other VMs in the VMHs are stili running. Thus, the workload of 
POWMHs is not static; migration of VMs may cause the workload of the target VMHs to much worse. 

Therefore, an effective load balancing mechanism is essential yet difficult for the management of 


VMHs. 


Load-balance is a typical but essential research topic in parallel or distributed systems, and many 
machine learning-based methods have been proposed, such as [10]— [12]. The following discusses 
some recent methods for load balancing in cloud computing. Alonso-Calvo et al. presented in [13] 
an application-level load balancing system for applications running on VMs. Doong et al. presented 
in [14] a multi-kernel support vector regression model for modeling VMs performance. Hu et al. 
developed a scheduling strategy for load balancing of VM resources using GA that refers to 
historical data and the current state of the system [15]. A capacity allocation algorithm is presented 
in [16] that coordinates multiple distributed resource controllers executing in geographically 
distributed cloud sites. Pang et al. presented in [17] a hybrid method that employs the estimation of 
distribution algorithm to estimate possible solutions of VM loads and then uses GA to adjust these 
solutions. HEELS [18] is a heuristic task deployment approach based on clustering and Glowworm 
Swarm Optimization and used for long-term load balancing of a cloud framework with edge 
computing. A hybrid metaheuristic is proposed in [19] that hybridizing artificial bee colony and ant 
colony optimization for load balancing of VMs in the cloud. 


These methods consider the load balance problem as job-assignment optimization and mainly focus 
on developing optimization algorithms for fast convergence. However, they assume the load of 
VM/VMH is static and do not consider the cost of migration and the load of VMHs after balancing, 
resulting in limited effectiveness in real environments. 

B. Motivation 


Based on the above discussions, an effective and efficient load balancing mechanism must consider 
the following. A load balancing mechanism works in a dynamic environment consisting of many 
running VMs and VMHs. It needs to monitor, detect, and decide how to release the imbalance 
situation of VMHs. An effective load balancing mechanism can take actions that release an 
overloaded VMH and do not result in overloads on another. Also, efficiency and low overhead are 
required for a balancing mechanism. Simple balancing methods like fixed or heuristic-based rules 
work efficiently but not effectively since they may derive overloading of other VMHs. Advanced 
balancing methods like statistics-based can work effectively; however, they need to calculate many 
sophisticated operational parameters and require considerable computation power to decide 
migration targets. 


Because there are diverse IaaS implementations, some assumptions are adopted in this study. 


1) AVM operates on only one VMH at a time, and multiple VMHs share a data storage. The 
amount of resource consumption of a VM cannot exceed the host VMH can offer. 


2) There are many types of load in computer systems, such as CPU/GPU computing load, 
memory/disk storage, network transmission. This study focuses on the CPU load of VMHs, 
which is mainly discussed in the literature. 


3) A centralized load balancing mechanism is employed for monitoring and managing all VMHs. 
The mechanism detects the number, locations, and all VMH workload parameters periodically. 


This study was motivated by the authors’ daily experiences in managing VMHs. Two genetic-based 
methods are developed and integrated for load balance. First, performance models of virtual 
machines (VMs) are extracted from their creating parameters and corresponding performance 
measured in a cloud computing environment. The gene expression programming (GEP) is applied 
for generating symbolic regression models that describe the performance of VMs and are used for 
predicting loads of VMHs after load-balance. Secondly, with the VMH loads estimated by GEP, the 
genetic algorithm considers the current and the future loads of VMHs and decides an optimal 
combination of VM-VMH assignments for migrating VMs and performing load-balance. Our 
proposed methods work effectively and efficiently for balancing VMH workloads in a dynamically 
working VMH environment. The performance of the proposed method is evaluated in a real cloud- 
computing environment. The experimental results show that our method outperforms previous 
methods. 


As stated in the literature, load balance is a critical issue for managing VMH servers in cloud 
computing. Using GEP, the idea of load prediction of VM for optimizing VM-VMH assignment helps 


Loading [MathJax)jax/outpuHTML-CSs/aurepaanntabee 2 bility of the load balance mechanism. The white-boxed GEP expression of VM load 
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„behaviors can be easily integrated with the balance mechanism. GA provides an intuitive and fast 

method of deciding VM-VMH assignments for load balance and flexibyy RQCAESarious 
ev iective functions for different management purposes. The genetic-based methods, GA and GEP, Q 
PO&an be easily implemented by the cloud administrator. The performance and usability of the 


proposed methods are evaluated and proved to be valid in a real cloud environment. 


TI 


SECTION II. 
Backgrounds 


A. Virtualization 


The virtualization technology is to establish a software layer called a hypervisor on a physical 
hardware platform. A hypervisor is an operating system that protects the resources of a VMH and PDF 
manages the requests and responses from VMs. Depending on the application environments, type- Help 
1 hypervisors directly running on VMH’s hardware and type-2 running on the VMH’s OS are 

defined. For more details about hypervisors, please refer to [20]. A VM is an abstract machine 

defined by virtual processors (vCPUs), virtual memory (vRAM), and virtual hard drives (vHDs) 

provided by a VMH. A user program running on a VM cannot directly access the physical 

hardware. Instead, all resources are wrapped by the VMH’s hypervisor. Generally, multiple VMs 

are executed on and managed by a VMH. Fig. 1 presents a collection of VMHs (referring to as a 

VMH server farm), where each VMH serves for one or many VMs. Commonly used hypervisors 

include VMware vSphere [21], KVM(kernel-based virtual machine) [22], and Microsoft Hyper-V 


[23]. 


FIGURE 1. - VMs, hypervisor, and a server farm with $m$ VMHs. 


FIGURE 1. 
VMs, hypervisor, and a server farm with m VMHs. 


Conceptually, a VM is composed of a “configuration file” describing the specifications of vCPU, 
vRAM, and vHD and a “disk image file” retaining the user data (vHD). A volatile “memory page” is 
generated in the VMH’s main memory when the VM is running. When a VM is created, the above 
files are created on the VMH’s storage. Therefore, deleting, copying, or moving a VM means 
deleting, copying, or moving these files. One of the advantages of virtualization is the easy migration 
of VMs, which means that one VM can move from one VMH to another VMH. With migration, VM 
can execute on different VMH platforms. During the migration, if the VM is in execution, the VM 
needs to be suspended first. In addition to moving the VM files (configuration and disk image), the 
memory pages associated with the VM must also be moved from the current VMH’s main memory 
to the target VMH. After all the movements are completed, the VM can be activated again. 


During migration, a VM may be suspended for seconds, even dozens of minutes, depending on the 
sizes and storage methods of VHD and vRAM. Additionally, migrating a VM incurs the transfer of 
vRAM pages over the network. The larger the vRAM, the more considerable the amount of data to 
be transferred, and the greater the system’s burden. The so-called “live migration” [24] is a fast re- 
configuration of VHD and vRAM for migration with a shorter suspending time with which the VM 
users may not even notice the suspension in the migration process. Live-migration is commonly 


i ith shared storage. VM migration can be easily and quickly done by re-assigning the 
Loading [MathJax]/jax/output/HTML-CSS/autoload/mtable.js ` r i y 5 . j 
disk mapping files instead of physically transferring vHDs (huge sizes). Fig. 2 
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illustrates the scenario of live migration with shared storage between two VMHs. Suppose the 
administrator selects VM4 to migrate. The disk mapping files associated Reantents.. reconfigured 
De¥i the storage, and the memory pages are transferred from VMH1 to VMH2. With live migration, 


Q 


PDfoad balance can be more efficient and reduce the impact on working VMs [25], [26]. 


= TI 
#.FIGURE 2. - Live migration with shared storage. 
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FIGURE 2. 
Live migration with shared storage. 
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The load of a VM is dominated by factors like CPU usage, memory size, network speed, etc. Assume 

that v is a VM, and L_V(v) represents its load. The L_V(v) can be described as a function, as shown 

in Eq. (1).\begin{equation*} L_V(v)=f(u_1, u_2, {\dots }, u_k),\tag{1}\end{equation*} 


View Source 


where u_1, u_2, {\dots }, u_k are the aforementioned k factors that dominate L_V . A VMH may 
serve many VMs. The workload of a VMH is the combination of the load from all its VMs and its 
operating system. Assume h is a VMH with n VMs (v_i, v_2, {\dots }, v_n ) running on h . The load 
L_H(h) ofh can be described as follows.\begin{equation*} L_H(h)=L_o(h)+\sum _{i=1}“n 
L_V(v_i),\tag{2}\end{equation*} 


View Source 


where, L_o(h) is the system load of h and L_V(v_i), 1\leq i\leq n, is the load of the VM v_i. 
Usually, each VM’s load is protected by SLA and should be above a reasonable level to maintain 
good service quality. Also, by the quota contract, VM’s load is limited. Eq. (3) describes the upper 
and lower bounds of L_V(v_i) .\begin{equation*} ]_ *(v_i) \leq L_V(v_i) \leq 1** 
(v_i).\tag{3}\end{equation*} 


View Source 


Usually, |_*(v_i) and 1**(v_i) are constants set by the administrator of VMHs according to the SLA 
and contract requirements. If L_V(v_i)\geq 1**(v_i) , the v_i’s load is too high, and it may not 
perform the user’s tasks smoothly. If L_V(v_i)\leq 1_*(v_i) , the load is low, causing a waste of 
resources of the VMH. 


Suppose there are p VMHs, h_1, {\dots }, h_p, operating in a server farm. The overall load of each 
VMH must also be within a reasonable range and can be described as Eq. (4).\begin{equation*} 
L_*(h_j) \leq L_H(h_j) \leq L**(h_j),\tag{4}\end{equation*} 


View Source 


where 1\leq j\leq p . Also, L_*(h_j) and L^*(h_j) are control parameters set by the administrator. 
If L_H(h_j)\geq L**(h_j) , the VMH is overloaded, and the VMs running on h_j may not obtain 
Loading [MathJax]/jax/output/HTML-CSS/autoload/mtable.js | 
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„ sufficient resources, degrading the performance and even service quality. If L_H(h_j)\leq L_*(h_j) 


, the computing resources are idle and wasted. := Contents 


Downl 

PDF 
With the above equations, to keep the load of a set of VMHs balanced is to keep all L_H(\cdot) s 
evenly, i.e., \begin{equation*} L_H(h_1)\approx L_H(h_2)\approx {\dots } \approx a 
L_H(h_p),\tag{5}\end{equation*} 


View Source 


and to preserve the consistency of Eq. (4) for all VMHs. If Eq. (5) is corrupted due to the overload 
of some VMHs, some VMs associated with these VMHs may need to be turned off or migrated to 
other VMHs for load-balancing. The load of all VMHs is expected to be balanced after the 


migration of VMs. Suppose h_s is a VMH about to become overloaded and h_t is a VMH in safe PDF 
load, i.e., L_H(h_s) > L**(h_s) and L_H(h_t)\ll L^*(h_t) . If migration of q VMs, v_1, {\dots }, 
v_q, from h_s to h_t is performed, the load of h_s and h_t changes as follows. \begin{align*} Help 


L'_H(h_s)=&L_H(h_s)-\sum _ {i=1}*q L_V(v_i),\tag{6}\\ L'_H(h_t)=&L_H(h_t)+\sum 
_{i=1}^q L_V(v_i)\tag{7}\end{align*} 


View Source 


Therefore, load balancing by the migration of VMs can be considered as a combinatorial problem, 
wherein the best set of VMs are selected from VMHs and migrated to proper hosts and Eq. (4)- 
Eq. (5) are always observed either before or after the VM migration. The following issues are 


considered in this study. 


e The load of VMHs is expected to remain balanced after the migration of VMs. It is necessary 
to predict the load of VMHs to prevent the target hosts from becoming overloaded after the 
migration. One method to obtain the descriptive expression of L_V(v_i) in Eq. (2) is to collect 
a sufficient amount of data and find a statistics or regression model accordingly. There are two 
types of learning methods, black-box and white-box [27]. The main difference between them 
is the explainability and readability of the regression model obtained. An explainable VM load 
model can be integrated and adjusted more easily with the management system from the 


system administration perspective. 


e Choosing the best set of VMs and target hosts for migration is a job-assignment optimization 
problem, usually a time-consuming combinatorial explosive problem. An efficient and effective 
selection mechanism is needed for load balance and painless migration. 


SECTION III. 
Genetic Methods 


There are two goals in this study. The first one is to describe the load of VMs as a white-box model. 
The second one is to decide on the best VM-VMH assignment for migration. This research adopts 
two genetic methods, genetic algorithm and genetic expression programming, for achieving these 
goals. 


A. Genetic Algorithm 


The genetic algorithm (GA) [28] is a stochastic search optimization method that mimics natural 
genetic systems’ mechanics. With GA, solutions to domain problems are encoded as chromosomes 
that are iteratively processed by genetic operators, reproduction, mutation, crossover, etc. The ones 
with better fitness are retained. GA is well known for its efficient exploitation of better solutions via 
search in the vicinity of known ones. GA has many advantages over traditional search methods: (1) 
it provides a simple and direct representation of solutions; (2) it searches solutions with a 
population of potential ones; and (3) it works with probabilistic transition rules instead of 
deterministic ones. GA has been successfully applied to many combinatorial optimization problems 
and is widely used in many applications. Genetic programming (GP) [29], is a genetic method for 
evolving programs or mathematic formulas. GP uses a tree structure to represent a program or 
formula, as shown in Fig. 3. The genetic operators in GP are similar to those in GA (mutation, 
crossover, and reproduction); however, they are applied as node-renaming, pruning, and 
recombination of subtrees. Because tree structures are complicated and computationally inefficient 
Loading [MathJax)jax/outpuHTML-Css/alraldaapherbiey’ 2 tion, some improvements are proposed, such as the following. 
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FIGURE 3. 
GP trees. 


GA is a type of metaheuristic algorithms that applies specific search schemas for solving complex 
optimization problems. Each metaheuristic algorithm employs a unique representation for problem 
description and a specialized mechanism for exploring the search space and is useful for particular 
applications. GA is adopted in this study due to the following reasons. 


e GA provides a direct and intuitive representation for describing the VM-VMH assignment and 
is easy to implement. 


e Many metaheuristic algorithms are working on an established objective function (fitness 
function), which may be changed for various load balance considerations (see Section VI). 


e For various optimization subjects, only the objective function’s calculation needs to be changed 
without massively changing the gene expression and the whole balance mechanism. 


More comparisons on the difference and usage of metaheuristic methods are available in [30], [31]. 
B. Genetic Expression Programming 


Genetic expression programming (GEP) [32] is a variation of GP. Given a set of terminals (input 
variables or constants) and operators (functions), GEP describes programs or formulas as binary 
trees represented in linear gene expressions. The gene expression is the level-order traversal of a 
binary tree and is composed of head and tail. Symbols in the head can represent functions or 
terminals; the ones in the tail can only be terminals. The length of a gene expression is h+t , where 
h is the length of head given by the users and t=h\times (n-1)+1 the length of tail. The parameter n 
is the number of most arguments of a function, also given by the users. Referring to Fig. 3(a), 
symbols x,y and integers are terminals and +, —, \times , \div , \sqrt { } are operators. A GEP tree 
with h=7 and n=2 is presented in Fig. 4, wherein the gene expression of length 15 (t=8 ) expresses 
(3x+8)\times (\sqrt {y}/2) . Notice that the GEP expression is determined by level-order traversal 
of its binary tree. Some terminals in the tail part may be useless. They are used as operands only if 
their parent nodes are operators. The genetic operators in GEP are reproduction, mutation, 
transposition, and recombination, applying on the gene expression. Reproduction and mutation 
are the same as those in GA. Transposition is to prune a gene segment and insert it to a position 
randomly selected. Recombination is similar to crossover in GA by breaking and recombining two 
gene expressions. The use of expression trees brings efficiency to GEP because genetic operations 
can be more easily applied in a simple linear structure. The cost for search specific gene elements 
in a genetic operation is reduced from O(n) to O(1) . GEP has superior performance in dealing with 
complex problems than conventional regression methods; for example, modeling sensor 
characteristics [33], diagnosis and prediction of lung cancer [34], fault diagnosis of power 
transformers [35], and intrusion detection of power grid [36]. Refer to [32] for more details 
about GEP. 
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FIGURE 4. 
A GEP example-gene and tree expressions. 


SECTION IV. 
The Proposed Methods 


A. Symbolic Load Models by GEP 


Symbolic regression is to find a description model from the input-output tuples of a problem that 
can rationally predict the correspondence of a new input. By Eq. (1), suppose the load of a VM v is 
determined by k parameters of resource settings, u_1, {\dots }, u_k . Let U=\{\langle u_{1i}, 
{\dots },u_{ki},y_i\rangle \} and y_i be the load of v , measured with the settings of u_1, {\dots 
},u_k at the i th instance. Suppose \tilde {f}(u_1, {\dots },u_k) is a symbolic regression model 
describing the relation of y and u_1, {\dots },u_k obtained from U . If the difference, usually 
measured by MSE (mean-square error), between \tilde {f}(u_{1i}, {\dots },u_{ki}) and y_i is 
acceptable, the model can be used for describing and predicting the load of VMs.\begin{equation*} 
\mathit {MSE}(\tilde {f},U) = \frac {1}{n}\sum _{i=1}*n(y_i - \tilde {f}(u_{1i}, {\dots 
},u_{ki}))*2,\tag{8}\end{equation*} 


View Source 


where n is the size of U . When using GEP, terminals are numeric constants and resource 
parameters, u_1, {\dots },u_k, and the operators are mathematic calculations, as listed in Table 1. 
For obtaining a fast VM migration decision, the operators performing complicated calculations 
(and consequently require longer computational times) are assigned lower evolutionary priority. 


TABLE 1 Operators Used in GEP 


w.Table 1- Operators Used in GEP 
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„B. VM Assignment by GA 


‘= Contents 
posince there may be many VMs and VMHs working in the cloud, deciding an optimal VM-VMH 


pp@ssignment is combinatorially explosive and time-consuming. Below we show how GA is applied for Q 
fast and reliable VM-VMH assignments. 


TI 


1) Chromosome Encoding 

Suppose that there are q VMs, v_o, {\dots }, v_{q-1} , served by p VMHs, h_o, {\dots }, h_{p-1} , 
in a VMH server farm. A chromosome consisting of q genes is defined as follows.\begin{equation*} 
[\varsigma _ 0, \varsigma _1, {\dots }, \varsigma _{q-1}],\tag{9}\end{equation*} 


View Source 


where \varsigma _i, 0\leq i\leq q-1 , denotes that the i th VM is served by the VMH h_i, 
\varsigma _i\in \{h_o,h_2, {\dots },h_{p-1}\} . A chromosome of the form in Eq. (9) represents a 
VM-VMH assignment. For example, Fig. 5 presents the assignment of 12 VMs to 3 VMHs. Help 


PDF 


#-FIGURE 5. - Chromosome encoding for a VM-VMH assignment. 


FIGURE 5. 
Chromosome encoding for a VM-VMH assignment. 


2) Load Estimations 


For the load balancing of VMHs by VM migration, a good configuration should correct the 
currently imbalanced situation. It will not result in a new imbalance after the migration of VMs. 
Suppose the chromosome in Eq. (9) is considered. Let \xi__o be the VM-VMH assignment before 
migration and \xi a new assignment generated by GA. Let \xi (v_i)=h_i denote the VMH assigned 
to v_iin \xi and \xi “{-1}(h_i)=\{v_j|\xi (v_j)=h_i\} the set of VMs assigned to h_iin \xi. By Eq. 
(2), the load of h_i is estimated as \begin{equation*} \tilde {L}_H(h_i) = L_o(h_i)+\sum 
_{j=1}%*{\tilde {L}_V(v_j)},\tag{10}\end{equation*} 


View Source 


where v_j\in \xi *{-1}(h_i) and \tilde {L}_V(v_j)=\tilde {f}(u_1, {\dots },u_k) is the load 
predicted by the regression model presented in Eq. (8). 


3) Migration Cost 

The cost of migration is also a critical issue that should be considered. Migration is costly, i.e., the 
time/resources consumption for retaining and moving VM data and the profit loss due to service 
suspension. Therefore, for a new VM-VMH assignment, the fewer moving VMs, the better. Let 
\theta be the cost of moving a VM, the migration cost \alpha (\xi) of a chromosome \xi is to 
examine \xi (v_i) , o\leq i\leq q-1, against \xi _o , i.e., \begin{align*} \alpha (\xi)=&1-\frac {1} 
{q}\sum _{i=o}*{q-1} \gamma (v_i),\tag{11}\\ \gamma (v_i)=&\begin{cases} 0, & \xi (v_i)=\xi 
_ 0(v_i),\\ \theta, & \xi (v_i)\neq \xi _o(v_i),\\ \end{cases}\tag{12}\end{align*} 


View Source 
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„ administrator. In a configuration with o migrating VMs, the migration cost is 0; otherwise, the 
migration cost is accumulated as the number of migrating VMs increases ONRERS of VMs is 
Costly, even with shared storage (live-migration). However, migration of VMs is a need for the 
dministration of VMH servers from energy-saving or SLA aspects [37]— [39]. A higher \theta 
setting reduces the occurrence of migration and eases the impact or inconvenience resulted from 
migration. Setting a reasonable \theta can refer to the aspects stated in [24], [40]. 
4) Fitness Function 
The fitness of \xi is defined by considering the migration cost \alpha (\xi) and the balance factor 
\beta (\xi) . The balance factor \beta (\xi) considers the load difference of two VMHs and 
computes the balance ratio against the maximum workload predicted.\begin{equation*} \beta 
(\xi)=\In \left(\frac {\frac {1}{2}p(p-1)\max _ {\forall }}(\{\tilde {L}_H(h_j)\})} {\sum _{\forall 
a\neq b}{|\tilde {L}_H(h_a) - \tilde {L}_H(h_b)]}}\right).\tag{13}\end{equation*} 


View Source 


The overall fitness of \xi is defined as follows, \begin{align*} fitness(\xi)= \begin{cases} \alpha 
(\xi)\cdot \beta (\xi),&L_*(h_j)\leq \tilde {L}_H(h_j)\leq L**(h_j),\\ &\mathrm {for~ all }~j,\\ 
o, & \mathrm {otherwise} \end{cases} \\ {}\tag{14}\end{align*} 


View Source 


Because the load of VMHs is expected to be “balanced” after migration, the fitness of a 
configuration is bad (0) if any VMH is predicted to be overloaded (> L^* ) or underloaded ( < L_* 
). Otherwise, the migration cost (\alpha (\xi) ) and the balancing degree (\beta (\xi) ) are 
evaluated. 


C. Load-Balancing Procedure 


Based on the proposed genetic-methods, an intelligent load balancing mechanism is implemented 
with two main procedures. Suppose there are q VMs running on p VMHs, the parameters 
associated with the generation of a GEP-based load model include: historical VM load data or the 
log records periodically collected by VMHs: \mathbf {D} , a linear GEP chromosome \zeta _0 
converted from the current load model, a set of operators: \mathbf {O} , population size: k , best 
GEP trees to retain: t , MSE threshold: \epsilon by Eq. (8), and head (\mathbf {h} ) and the 
maximum number n of operators in a GEP tree. The pseudocode of the GEP-based load model 
generation is presented in Algorithm 1. The load-balance procedure is invoked periodically with 
the following parameters. The current VM-VMH assignment \xi _o=[\varsigma _0, \varsigma _1, 
{\dots }, \varsigma _{q-1}] in the form of Eq. (9), Load-Balance Status (LB) by Eq. (13), 
Population size: K , Best chromosomes to retain: T , Best fitness threshold: B=M\times fitness(\xi 
_0) . The pseudocode of our proposed GA-based load balance mechanism is presented in 
Algorithm 2. 


Algorithm 1 The GEP-Based Load Model Generation 
1: function GEP_ Load_ Model_ Generator 


2: Compute tail size t=h\times (n-1)+1 ; 
3: Evaluate MSE \epsilon _o of \zeta _o against \mathbf {D} ; 
4: while (\epsilon _o < \epsilon ) do 


5: Randomize k GEP chromosomes \zeta _i, 1\leq i\leq k , \zeta _i is of length h+t (as in Fig. 
4(a)); 


6: Set the initial population as R=\cup _{i=o0}”k\{\zeta _i\} ; 
7: Evaluate MSE \epsilon (\zeta _i) of \zeta _i in R by Eq.(8); 
8: repeat 


9: Retain top-T elements of R with lower MSE; 
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,10: Perform GEP operators on \zeta’s; me 
B := Contents 


Downl 
ppt Evaluate MSE \epsilon (\zeta _i) of \zeta _i by Eq.(8); Q 
12: Select the best element \zeta ' as the best VM load model and \epsilon _o=\epsilon (\zeta ') ; of 


13: until (termination conditions: max. iteration or low-enough MSE) 
14: end while 

15: return GEP expression of \zeta ' as the VM load model; 

16: end function 


PDF 


Algorithm 2 The GA-Based Load Balance Procedure Help 


1: procedure GA_Balance_Procedure 
2! Evaluate load of each VMH, L_{H}(h_{i}) , o\leq i\leq p-1; 


3: By LB, evaluate load balance status al all VMHs; 


4: while (LB < 0 lasts for 30 sec.) do 

5: Randomize K chromosomes, \xi _ {i} , 1\leq i\leq K, all in the form of Eq. (9); 
6: Set the initial population as S=\bigcup _{i=0}”*{K}\{\xi _{i}\} ; 

7: Evaluate the fitness of all \xi _{i} in S by Eq.(14); 

8: repeat (on S ) 

9: Retain top-T elements with best fitness; 

10: Perform crossover/mutation on \xi’s; 

11: Evaluate the fitness of each \xi _ {i} by Eq.(14); 

12: until (termination conditions: max. iterations, time limitation, high fitness) 
13: Select the best chromosome \xi' as the new VM-VMH assignment if fitness(\xi ')\geq |B| ; 
14: Perform migration by \xi' ; 

15: end while 


16: end procedure 


Notice that the current VM load model \zeta _o and the current VM-VMH configuration \xi _o are 
also include the initial population of GEP_Load_Model_Generator and GA_Balance_Procedure, 
respectively. Such settings make sure that solutions better than the current one can be derived. 
Initially, the training data for the VM load model can be manufactured, as shown in Section V-B. 
They are collected from VMH’s log and used to update the VM load model, as shown in 
GEP_Load_Model_Generator. Therefore, as the system performs, the VM load model evolves 
reasonably to the real situation. GA_Balance_Procedure performs load balance when an imbalance 
is detected. It searches by GA a new VM-VMH assignment better than the current configuration 
and suitably re-balance the system. The two procedures work individually and periodically, as 
depicted in Fig. 6. 
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Load monitoring and balancing process. 


SECTION V. 
Experiments 


A. Environment and Parameter Settings 


The proposed methods were implemented and verified in a small-scale but real cloud environment, 
Jnet. Jnet is a private intranet, consisting of more than 20 various types of servers accessed by 
about 200 users. The VMHs and the load balance monitor were attached to Jnet. The experimental 
environment consisted of four VMH servers, including three HP ProLiant BL460c blades and one 
HP ProLiant ML350, all running in CentOS. The shared storage server was an HP ProLiant ML110 
running FreeNAS. All machines were connected by 1GB ethernet. KVM is the platform for VMHs. 
The centralized load balancing mechanism was executed and controlled by the VMHo node. The 
architecture and specifications are shown in Fig. 7. Because the resource capacity of each VMH is 
different, the number of VMs that a VMH can support is also different. In the following 
experiments, the VM capacity of VMHo, VMH1, and VMH2 is 48, and the capacity of VMH3 is 12. 


There are in total 48 VMs running in each experiment. 


FIGURE 7. - Experimental environment settings. 


FIGURE 7. 
Experimental environment settings. 


A VMH manages the resource consumption of VMs by setting capacity parameters that restrict the 
number of resources used by VMs. Such parameters are contract-protected and adjustable. In this 


study, two parameters of CPU utilization are discussed. 
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. ° CPU utilization: This parameter explicitly limits the CPU consumption of a VM and restricts 
+] the VM’s performance/load. There are various implementations-6F ste Bhentsters on 


Downl! 


different virtualization platforms. The one used in this study is cpu.cfs_quota_us defined by Q 
POF the Linux Cgroups [41], which ranges from 1000 to 100000, representing the CPU utilization 
from 1% to 100%. 
TT 
e Usage priority: This parameter defines the priorities of VMs for using a CPU core. Also, the 
parameters cpu.shares defined be the Linux Cgroups is used in this study, which ranges from 1 


to 65535. 


This study defines two parameters for evaluating the performance of the load balancing methods. 
e Load-Balance status (LB). LB is the value of Eq. (13) applied to the current VM-VMH 
configuration, the more balanced load of VMHs, the higher LB. 
e Balancing Efficiency (BE). BE is the number of load balancing iterations performed by the PDF 
monitor to make all VMHs balanced (LB>o) after migration of VMs; the fewer rounds, the Help 
faster the load balancing efficiency. 


The balancing and monitoring mechanisms were implemented in C++ with libvirt [42] as the 
software interface. Each VMH scans and logs its CPU load by Eq.(2) every 10 seconds. A load 
imbalance situation is detected if LB<o lasts for the past 30 seconds. Load limitations of a VM are 
L^*=25 % and L_*=o %. The length of a chromosome is the number of VMs currently served in the 
data center. In the following experiments, six load balancing heuristics are compared with our 
proposed one. 


1) LtoS: move the top-loaded VM from over-loaded VMHs to the VMH with the lowest load. 

2) LtoR: move the top-loaded VM from over-loaded VMHs to a random VMH. 

3) StoS: move the lowest loaded VM from over-loaded VMHs to the VMH with the lowest load. 
4) StoR: move the lowest loaded VM from over-loaded VMHs to a random VMH. 

5) RtoS: move a random VM from over-loaded VMHs to the VMH with the lowest load. 


6) RtoR: move a random VM from over-loaded VMHs to a random VMH. 
B. Modeling VM Loads 


The first experiment is to derive load models of VMs. For training GEP, a set of data in the form of 
\langle cpu.cfs_quota_us, cpu.shares, L_V(\cdot)\rangle was collected. Because there is no 
historical VM load data, we have to generate load data similar to real VMs. A VM v^* was designed 
for collecting load data with a load generator. The generator performed several computation- 
intensive and I/O-intensive instructions. The computation-intensive part is to calculate 
sin((double)rand()/RAND_MAX) and the I/O-intensive part is to read data from /dev/zero and 
write to /dev/null. The program was randomly executed and lasted for 10 seconds, the 
computation or I/O instructions on v** . The parameters, cpu.cfs_quota_us and cpu.shares were 
randomly set for v^* ; cpu.cfs_quota_us was set 10000—100000 and cpu.shares was partitioned 
into 1024 levels. A total of 1024 combinations of parameters were generated, each used to initialize 
v^* . The VM v^* executed on a VMH for seven times, and the load L_V(v**) was measured. A 
total of 7168 load data were collected. Then, the operators listed in Table 1 and the hyper- 
parameters listed in Table 2 were used with terminals cpu.cfs_quota_us and cpu.shares for 
symbolic regression of L_V(v**) . 


TABLE 2 GEP Hyper-Parameters 
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To verify the effectiveness of the GEP models, some regression methods [43], polynomial(POL), 
linear(LIN), exponent(EXP), and power(POW) regression, were used to analyze the same VM load 
data and compared it with GEP. Variable portions of the training data were randomly selected for 
training, and the rest portions were used for validation. The same training process was repeated ten 
times. The training results are presented in Fig. 8. 


FIGURE 8. - GEP training results with 10%-90% training data. 


FIGURE 8. 
GEP training results with 10%-90% training data. 


Because the best model was with 90% of training data, the same data set was also applied to other 
regression methods. The load models generated by various regression methods are as follows: 


e POL: -0.166 + 0.0939 u_1 + 0.0001 u_1^2 - 0.00004 u_2 + 0.00000001 uU_2^2 
e LIN: -0.3235 + 0.103 u_1 - 0.00002 u_2 
e EXP: \exp (-0.4866 + 0.0337 u_1 - 0.00004 u_2) 


e POW: 0.0394 u_1*{1.2216} u_2”{-0.0004} 


GEP: \In ((1 / u_2) - 0.06 u_1^2) 


For simplicity, in the above models, cpu.cfs_quota_ us $= u_1$ and cpu.shares $= u_2$. Fig. 9 

and Table 3 present the testing results of all regression models. In terms of MSE and MAE, GEP, 
POL, and LIN perform competitively, and EXP is the worst. MAPE indicates that GEP, POL, and 

POW are more stable than others. POL and GEP produce similar error rates; however, POL uses 

more and complicated operators than GEP does. LIN uses the fewest operators but is more 


Loading [MathJax] escaping c/n ARIBIE}E mS of MSE/MAE and unstable in MAPE than GEP. GEP produces a much compact 
formula than others and can be considered competitive. 
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# FIGURE 9. - Statistics on errors of all models. 


FIGURE 9. 
Statistics on errors of all models. 


C. Load Balance Scenarios 


In the following experiments, the VM load model generated by GEP was used for load prediction. 
The load balancing monitor of Fig. 6 was applied to four scenarios. A total of 48 VMs were 
deployed, each of which performed similarly as v^* . In the following tests, 40 runs of load 
balancing were observed. VM migration was performed when the load was imbalanced, or the GA 
produced a VM-VMH assignment with LB 1.25 times better than the current configuration. The 
hyper-parameters associated with GA include: max. generation: 1000, population size: 30, 
crossover_rate: 0.7, mutation_rate: 0.4, time limitation: 60 sec. 


1) Scenario A 

In this scenario, an extreme imbalance is simulated. In the beginning, all VMs were gathered 
together on a randomly selected VMH, making the VMH heavily overloaded. Fig. 10 plots the 
change of load of all VMHs in 40 rounds of load-balancing. Table 4 lists the loads of all VMHs, 
where R2R is the average of RtoRi and RtoR2 ( Fig. 10(g)(h)). From Fig. 10, it is observed that in 
the first round, all VMs are concentrated to a VMH, the VMH loads are extremely imbalanced, and 
thus, the LB value is o. However, our proposed method takes only one round to re-balance the loads 
for all VMHs. The workloads of all VMHs approximate evenly after load-balancing, which can be 
observed in Table 4. Other strategies take more iterations for re-balancing the loads. In the cases 
using LtoS, StoS, and RtoS, LtoS migrates the most heavily loaded VMs to the lowest loaded VMH 
and reaches a balanced state in 20 rounds, causing load vibration before back to balanced. Though 
RtoS takes 35 rounds for re-balancing and StoS even not back to balanced, the load of VMHs trend 
to balance. Other strategies even not back to the balanced state. 


TABLE 4 Comparisons on Average Load (%) in Scenario-A 
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>FIGURE 10. - Loads and LB trends in Scenario-A. 


FIGURE 10. 
Loads and LB trends in Scenario-A. 


Another extremely imbalance is studied in this experiment, wherein a temporary service suspension 
of a VMH happens so that all its VMs have to be migrated to other VMHs for non-stop services. In 
this scenario, all VMs on VMHi are moved to VMHoO at round o, and VMHi backs to service at 
round 1. The balancing status is observed in Fig. 11. It is found that VMHo is overloaded, and VMH1 
is idle in the beginning. Again, our proposed method takes only one round to re-balance loads of all 


VMHs and keep them almost balanced in the consequence rounds. Some strategies can also re- 
balance the loads, such as LtoS, LtoR, StoS, and even RtoR; however, they take more rounds to re- 
balance and only manage a low-level balanced status. This phenomenon can be observed in Table 5. 


TABLE 5 Comparisons on Average Load (%) in Scenario-B 


Zable 5- Comparisons on Average Load (%) in Scenario-B 
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FIGURE 11. 
Loads and LB trends in Scenario-B. 


3) Scenario C 

This experiment simulates the daily use of VMs-—the turning on/off of VMs at peak/non-peak 
hours, wherein more VMs are online/offline in the peak/non-peak hours. Thus, VMHs are less- 
loaded at non-peak hours and vice versa. The timing of turning on or off a VM is decided by a 
Gaussian distribution model, with \mu =10 and \sigma =5 for turning on VMs and \mu =20 and 
\sigma =5 for being online before turning off. That is, a VM starts at around the 10th round and 
stays online for about 20 rounds, simulating the 20th round as the peak hour and the ist and 40th 
rounds the off-peak hours. The results are presented in Fig. 12. Notice that, due to no fair 
comparison on two VMH load that is stochastic in this scenario, average load of VMHs is not 
presented. Obviously, the VMH load in this scenario is highly dynamic, which challenges the load- 
balancers. In the beginning, all methods do not balance VMH load effectively due to the 
unpredictable behaviors of VMs being online, and all LBs are o. It is found that all methods can 
balance loads to some extend in peak hours. This may be because the load behaviors become stable 
in the peak hours, with more VMs being online so that the load status is more easily to remain after 
VM migration. The LtoR strategy is the first to re-balance the load; however, it only works in the 
non-peak hours (the 5th round) when there are only a few VMs. Some strategies, including LtoR, 
LtoS, StoR, do not balance loads effectively in the peak hours and even failed after the peak hours. 
Our proposed method demonstrates a better performance on load balancing than others. 


FIGURE 12. - Loads and LB trends in Scenario-C. 


FIGURE 12. 
Loads and LB trends in Scenario-C. 


4) Scenario D 

Scenario D is an extended version of Scenario C, wherein the dynamics of VMHs is considered. In 
this experiment, the system load of a VMH, i.e., L_o(\cdot) , bursts intentionally, and the VMH is 
undoubtedly overloaded. Thus, the VMs served by this VMH have to migrate for load-balancing. 


eanne Ls behave like those in Scenario C; they turn on and off in peak/non-peak times. 
Loading [MathJax]/jax/output/HTML-CSS/: l le.j 
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„ Tespectively. Their load bursts continue for seven rounds and back tq service at the 14th and 34th 
rounds, respectively. The balancing results are presented in Fig. 13..Oer FERkePtiethoa is 
De*Ssnsitive to the imbalance in this highly dynamic environment and can re-baiance the VMH loads Q 
POFefficiently. Static heuristic rules, such as LtoS and StoS, are also sensitive and able to re-balance; 
they do not work well in all cases. Random heuristic rules may be workable; they are not stable and aT 


have poor performance. 
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FIGURE 13. 
Loads and LB trends in Scenario-D. 


Table 6 lists the averaged LB and BE values of all methods in the four scenarios. A higher LB value 
means a better-balanced performance. A lower BE value presents better balancing efficiency. Our 
proposed method demonstrates effectiveness and efficiency in either stable (e.g., scenarios A and B) 
or dynamic (e.g., C and D) environments in terms of LB and BE. With VM load models obtained 
from GEP, our method predicts the VMH loads after migration and computes by GA the most 
effective VM-VMH assignment for migration. Other strategies perform heuristic VM migration and 
take longer rounds for re-balancing, or even not back to balance. Interestingly, according to LB and 
BE in Table 6, LtoS is the second-best strategy. LtoS is intuitive and straightforward to implement, 
i.e., to move the worst VM to a VMH with the lowest load; however, it takes a longer time to re- 
balance VMH loads. Random methods, such as LtoR, StoR, RtoS, and RtoR, are effective sometimes 
but are not recommended due to their stochastic characteristics and unstable performance. Our 
proposed method performs effective and efficient load balancing in all tests. It is competitive and 
more promising than others. 


TABLE 6 Comparisons on LB and BE of all scenarios. 


Table 6- Comparisons on LB and BE of all scenarios. 


GEP_Load_Model_Generator updates the VM load model every 120 seconds periodically. The 

monitor GA_Balance_Procedure is activated every 250 seconds and calculates the LB value of all 

VMHs. Notice that the frequency of executing the two procedures should not be too high. Higher 
Loading [MathJax)jax/outpuHHTML-CSS/alerdatingaplegsbalancing frequency may keep the VM load model and the system as real and balance 
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„as possible. However, the operations of performance measurement and migration are costly: the 
BSsteal of CPU cycles degrades the overall performance and QoS, wheres fe Rabe Atsration may 
De¥aiuse unacceptable service suspension. Hence, the period of load monitoring is 250 seconds and the 
POF eriod of updating the VM load model is 120 seconds. 

Moreover, as in other machine learning algorithms, setting hyper-parameters associated with GA i 
and GEP is subtle and complex. The size of population, probabilities of genetic operators, and the 
termination condition are all problem-dependent. In general, higher probabilities of invoking 
genetic operators usually drive GA or GEP early mature but easily trapped in local optimal; lower 
probabilities take much time to converge. A large population consumes considerable memory space 
and computing time but may earn a higher chance of evolving a better solution. Due to the costs of 
performance probing and migration, deriving a solution (VM load model or VM-VHM assignment) 
should be timely. Such a solution may not be the best one but has to be acceptable and better than 
the current one. For example, a new VM-VMH assignment has to be 1.25 times better than the 
current one to be acceptable. Other hyper-parameters are set by observing the behaviors of VMs and PDF 
VMHs in JNet. Several combinations of parameters are attempted. For the length of this paper, Help 
some meaningful parameter sets are presented in the experiments. For issues of setting hyper- 


parameters associated with genetic methods, the readers can refer to [44], [45]. 


Regarding the efficiency of the balance mechanism, the main concern is how fast an effective VM- 
VMH assignment is suggested. The complexity of each strategy depends on how it is implemented. 
In this study, all strategies are implemented with standard C/C++ libraries. Searching for a 
min/max item can be done in \mathbf {O}(n) and generating a random number can be done in 
\mathbf {O}(1) , where n is the number of VMs. However, running a GA-based method is 
stochastic. Its complexity depends on the representation of chromosomes, the implementation of 
the genetic operators, and the fitness function. Some studies presented theoretic and experimental 
analysis on the complexity of GA [46], [47]. The complexity of our method is more likely \mathbf 
{O}(gpn) for the stochastic process and \mathbf {0}(n^2) for fitness evaluation, where g is the 
number of generations and p is the population size. In our experiment, g is 1000 and p is 30. The 
GA iteration is usually terminated by a low LB value and is less than g . When implementing our 
method in a large-scale cloud environment, the fitness evaluation could be the computational 
bottleneck, which can be implemented with a sophisticated data structure to reduce the 
computation cost. 


SECTION VI. 
Conclusion 


This paper presents a load balancing mechanism based on evolutionary computing. Loads of VMs 
and the associated resource parameters are measured and used for constructing symbolic regression 
models of VM using GEP. An optimal combination of VM-VMH assignment is decided by GA, which 
predicts VMH loads by GEP models and suggests the VMs be migrated for load-balancing. 
Experiments are conducted in a small-scale but real cloud environment. The proposed method 
demonstrates its effectiveness and efficiency on load balancing and is competitive and promising. 
Although the proposed method performs well for load-balance, it can be improved in some aspects 
and discussed. 


Currently, constant lengths of the head and tail of a GEP chromosome may limit the searchability of 
GEP in the solution space. The variable size of the head that is determined by the training data may 
produce better regression models. The initial VM load model is built from simulated data. However, 
according to the architecture depicted in Fig. 6, VM load data are collected continuously; the VM 
load model can also be updated accordingly. GEP and GA are the genetic-based methods employed 
in this study. It is possible to use other methods to generate regression models and decide the 
optimal combination of VMs for migration, provided that white-box regression models and fast 
decision-making can be derived. 


Due to the hardware resource limitations, the test environment only consists of four VMHs with a 
centralized load-balancer installed on one VMH. However, the usability of our proposed method is 
verified. The proposed method can be applied to a distributed, large-scale management architecture 
easily. At present, the CPU load of VMs and VMHs is considered. Other types of load, like 
networking bandwidth, memory data transfer rate, or GPU loads, may also be worth studying. 
However, these types of loads are stochastic, and their measurements take considerable overhead. 
They are not included in this paper but may be processed similarly by our proposed method. 
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„This study focuses on the load balancing of VMHs that demand VH§s to work evenly. There may be 


other VMH management strategies, e.g., power saving or utilization sexi ME MEA tH at can be done 


POFhat some VMHs can be turned off [48]- [50]. These can be easily done using our proposed 
methods by re-defining the fitness function of GA. The current fitness function and balance factor 
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are based on the ratio of the maximum load to the load differences among VMHs, which is simple 


but fast. Comprehensive statistics on the VMH loads may produce a sophisticated fitness function 


and LB formula [51]. Considering all these factors for load balancing is a multi-objective 
optimization problem, which requires sophisticated methods for maintaining computation 


efficiency. However, measurements on the detailed resource consumption of VMs may take much 


more computational resources and thus degrade the performance of VMHs and the load-balancer. 


The trade-offs between efficiency and measurement need advanced studies. Instead of GA, other 


metaheuristic algorithms may also decide VM-VMH assignment efficiently. GA is competitive than 


others in the practical administration of cloud computing servers because it can easily and flexibly 
adjust the optimization objectives and maintain a clear and intuitive representation for VM-VMH 
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assignment. Studies on the performance and applicability of various metaheuristic algorithms for Help 


different balance purposes are worthy. These will be part of our future work 
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